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PREFACE- 


This  research  paper  focuses  on  the  issue  of  legality  vis-a-vis  the  Air 
Force  Officer  Effectiveness  Report.  It  recognizes  the  increasing 
involvement  of  the  courts  in  corporate  performance  appraisal  systems,  and 
discusses  general  guidelines  the  courts  have  issued  that  assist  companies  to 
make  their  systems  more  legally  defensible.  The  author,  using  primarily  AFR 
36-10,  analyzes  the  extent  the  Air  Force  complies  with  these  court-issued 
guidelines.  In  areas  found  to  be  deficient  or  illegal  the  author  provides 
suggestions  that  would  make  the  OER  more  defensible  in  the  courts  if  it  were 
ever  challenged.  While  making  the  OER  more  legal,  these  suggestions  would 
have  the  immediate  benefit  of  making  the  OER  a  more  effective  management 
tool  . 
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EXECUTIVE  SUMMARY 

Part  of  our  College  mission  is  distribution  of  the 
students’  problem  solving  products  to  DoD 
sponsors  and  other  interested  agencies  to 
enhance  insight  into  contemporary,  defense 
related  issues.  While  the  College  has  accepted  this 
product  as  meeting  academic  requirements  for 
graduation,  the  views  and  opinions  expressed  or 
implied  are  solely  those  of  the  author  and  should 
not  be  construed  as  carrying  official  sanction. 


insights  into  tomorrow” _ 

REPORT  NUMBER  88-2200 

AUTHOR(S)  MAJOR  MICHAEL  S.  REESE,  USAF 
TITLE  THE  OFFICER  EFFECTIVENESS  REPORT— IS  IT  LEGAL? 


I.  Purpose:  To  determine  the  extent  of  compliance  of  the  Air  Force  Officer 
Effectiveness  Report  (OER)  with  court-issued  guidelines  that  assist 
companies  in  the  legal  design  and  implementation  of  performance  appraisal 
systems.  In  areas  found  to  be  deficient  and/or  illegal,  to  provide 
suggestions  that  would  make  the  OER  more  defensible  in  the  courts. 

II.  Problem:  The  trend  for  the  courts  to  become  increasingly  more  involved 
in  a  company's  performance  appraisal  system  is  a  recent  phenomenon.  As  a 
result,  the  corporate  world  has  dramatically  changed  the  way  they  evaluate 
their  employees.  Given  the  relative  stability  of  the  OER  over  the  same 
period,  it  is  likely  the  OER  would  be  difficult  to  defend  in  the  civil 
courts. 

III.  Data:  There  exists  a  core  collection  of  court  cases  that  adequately 
summarize  the  extent  of  court  Involvement  in  private  sector  performance 
appraisal  systems.  Through  these  court  cases,  the  courts  have  substantively 
affected  six  major  areas  within  performance  appraisals:  job-relatedness, 
job  analysis,  subjectivity  (or  conversely,  objectivity),  communication, 
training,  and  rater  characteristics.  Consequently,  management  consultant 
groups  and  academicians  have  published  numerous  guidelines  companies  can 
follow  to  make  their  employee  evaluations  more  defensible.  The  author  has 
reduced  these  guidelines  to  ten  operating  guidelines  that  particularly 
impact  the  Air  Force  OER.  They  provide  an  excellent  checklist  for  the  Air 


vi 


Force  to  use  to  determine  how  legal  its  OER  is.  An  important  internal  Air 
Force  document  that  must  be  considered  before  any  final  determination  can  be 
made  is  AFR  36-10.  As  the  sole  authority  on  officer  evaluations,  AFR  36-10 
is  the  definitive,  officially-sponsored  guidance  package  on  the  OER.  Only 
after  thoroughly  studying  AFR  36-10,  and  comparing  its  guidance  to  the 
court-issued  guidelines,  can  a  final  legal  assessment  be  made  concerning  the 
OER. 

IV.  Conclusions:  The  Air  Force  clearly  does  some  things  right, 
particularly  in  the  area  of  equal  opportunity.  However,  and  on  balance, 
much  work  is  required  if  the  Air  Force  is  to  better  defend  their  officer 
evaluation  from  a  legal  perspective.  Areas  receiving  marginal  or  failing 
ratings  and  require  significant  changes  to  current  policy  are 
job-relatedness,  validity,  and  rater  training.  Areas  receiving  satisfactory 
ratings  but  still  require  minor  changes  are  rater  characteristics, 
performance  review,  and  ratee  communication. 

V.  Recommendations:  The  US  Air  Force  should  begin  listening  to  the  free 
advice  the  courts  are  offering  on  how  to  better  construct  and  implement 
performance  appraisal  systems.  Significant  improvements  can  be  made  without 
affecting  the  design  of  the  OER;  others  require  a  complete  overhaul  of  the 
form  itself.  The  net  effect  of  compliance,  even  minimal  compliance,  with 
the  court-issued  guidelines  can't  help  but  improve  the  overall  effectiveness 
of  the  OER. 


Chapter  One 


INTRODUCTION 

For  the  past  decade  US  civil  courts  have  become  increasingly  more 
involved  in  a  company's  performance  appraisal  system.  Particularly,  tne 
courts  have  begun  to  question  the  legality  of  private  sector  evaluation 
systems.  Through  an  increasing  number  of  important  court  cases  in  this 
area,  companies  now  face  extremely  important  legal  questions.  Are  our 
performance  appraisal  systems  defensible  in  tne  courts?  Are  tney.  in  fact, 
legal?  This  paper  will  address  these  questions  relevant  to  one  particular 
company — the  United  States  Air  Force--and  to  its  performance  appraisal 
system— the  Officer  Effectiveness  Report  COER). 

Certainly  there  has  been  recent  talk  on  the  problems  surrounding  tne 
OER.  In  fact,  a  recent  retired  Air  Force  general  speaking  to  tne  Air 
Command  and  Staff  College,  and  to  no  ones  surprise,  labeled  tne  entire  OER 
"bankrupt."  Even  at  this  writing,  a  special  working  group  at  tne  Military 
Personnel  Center  (MPC)  is  preparing  to  brief  the  Air  Staff  on 
recommendations  for  change  and  improvements  to  the  current  OER  system  This 
paper  is  not  an  attempt  to  supplement  their  work,  nor  refute  it.  Nor  will 
it  propose  a  new  and  independently  designed  OER  that  meets  every  legai 
requirement.  Rather,  it  is  the  paper's  goal  only  to  expose  the  current 
system  to  the  criticisms  and  recommendations  coming  from  US  civil  courts. 

It  is  purely  a  reflective  paper  that  will  answer  the  following  questions: 

How  defensible,  in  a  legal  sense,  is  the  current  Air  Force  Officer 
Effectiveness  Report?  If  it  was  legally  challenged  in  the  civil  courts,  now 
wou Id  it  hold  up? 

The  following  analysis  will  focus  on  three  major  areas.  First,  it  win 
summarize  recent  (post  1970)  court  decisions  that  have  had  an  impact  on  tne 
way  companies  evaluate  their  employees.  This  summary  evolved  from  a 
thorough  literature  search,  primarily  involving  business  and  legal 
periodicals  and  journals.  Second,  it  will  integrate  these  court  decisions 
into  a  set  of  guidelines  companies  should  follow  to  legally  defend  their 
performance  appraisal  systems.  These  guidelines  allow  companies  (and  the 
Air  Force)  to  conduct  a  quick  self-assessment  of  their  performance  appraisal 
system.  They  are  simple  benchmarks  of  comp  1  i ance—what  are  we  doing  ngnt 
and  what  our  we  doing  wrong?  Finally,  it  will  compare  these  guidelines  to 
the  way  the  Air  Force  manages  its  performance  apprai sa I --the  OER.  Air  Force 
Regulation  (AFR)  36-10  is  the  sole  authority  on  officer  evaluations.  By 
design,  the  major  commands  (MAJCOM)  offer  little  additional  guidance  on  now 
to  administer  the  OER.  Consequently,  AFR  36-10  wi i 1  be  the  primary  source 


to  identify  areas  in  which  the  Air  Force  meets  or  fails  to  meet  the 
court-issued  guidelines.  Coupled  with  this  comparison  are  suggestions  that 
would  make  the  current  OER  more  defensible.  These  recommendations  not  only 
have  design  implications  for  future  revisions,  but  have  practical 
suggestions  for  how  to  improve  the  administration  of  any  OER,  regardless  of 
des 


The  courts7  Involvement  with  corporate  performance  appraisal  systems  is 
a  recent  phenomenon.  The  impetus  for  their  involvement  came  from  the  US 
Congress,  the  Equal  Employment  Opportunity  Commission  (EEOC),  and  the 
Department  of  Labor.  "On  July  2,  the  US  Congress  passed  the  Civil  Rights 
Act  of  1964  [Title  VIII. “  (8:29)  Title  VII  simply  makes  discrimination 
based  on  race,  color,  religion,  sex,  or  national  origin  illegal.  (8:29)  In 
1972,  the  EEOC  and  the  Department  of  Labor  released  the  Uniform  Guidelines 
on  Employee  Selection  Procedures.  This  document  established  performance 
appraisals  as  a  selection  instrument.  "The  Uniform  Guidelines  do  not  apply 
only  to  written  tests;  they  cover  all  selection  procedures  that  are  used  in 
making  employment  decisions.  Therefore,  the  Uniform  Guidelines  are  clearly 
applicable  to  performance  evaluations  when  the  results  of  these  evaluations 
are  used  in  making  employment  decisions."  (8:32)  The  implication  behind 
these  guidelines  is  critical.  "The  courts  have  generally  judged  the 
legality  of  employment  procedures  under  Title  VII  according  to  the 
principles  laid  out  by  the  guidelines."  (8:32)  Consequently,  since  1972, 
courts  began  scrutinizing  companies7  performance  appraisal  systems  from  a 
discrimination  standpoint.  In  those  cases  where  the  courts  have  found 
companies  to  discriminate  through  their  performance  appraisal,  the  courts 
have  ruled  the  appraisals  1 1 1 eaa 1 . 


While  the  issue  of  discrimination  is  at  the  heart  of  a  company's  legal 
performance  appraisal  system,  it  has  a  much  broader  application.  "Whenever 
an  employee's  performance  is  reviewed  or  evaluated,  we  are  making 
discriminations."  (32:81)  This  attitude  is  affecting  the  court's  decision 
to  legally  assess  performance  appraisals  even  if  discrimination  is  not 
evident.  “Employees  must  be  concerned  about  defending  their  appraisals 
regardless  of  the  presence  of  adverse  impact  [discrimination]."  (22:109) 
This  is  important  when  discussing  the  Air  Force  Officer  Effectiveness 
Report.  The  issue  is  not  whether  the  Air  Force  discriminates  with  its  OER. 
The  courts  are  simply  providing  all  companies,  both  private  and  public, 
important  legal  guidance  on  how  to  design  and  administer  their  performance 
appraisal  systems.  This  guidance  is  not  only  intended  to  correct  current 
discrimination,  but  also  to  prevent  future  discrimination. 
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Chapter  Two 


PERFORMANCE  APPRAISALS  AND  THE  COURTS 

There  exists  a  core  collection  of  court  cases  that  do  an  adequate  job 
of  summarizing  the  extent  of  court  involvement  in  private  sector  performance 
appraisal  systems.  Without  getting  involved  in  the  actual  participants  and 
their  particular  arguments,  this  paper  will  provide  bottom-line  messages  the 
courts  are  sending  to  the  private  sector  (and  indirectly  to  the  US  Air 
Force).  There  are  six  major  areas  within  performance  appraisals  that  courts 
have  substantively  affected:  job-rel atedness ,  job  analysis,  subjectivity 
(or  conversely,  objectivity),  communication,  training,  and  rater 
characteristics. 


JOB-RELATEDNESS 

"The  Uniform  Guidelines  issued  by  the  Equal  Employment  Opportunity 
Commission.  .  .  make  it  clear  that  performance  appraisals  must  be 
job-related.  .  .  .“  (2:359)  To  be  job-related,  a  performance  appraisal  must 
evaluate  and/or  measure  the  behaviors  necessary  for  successful  job 
completion.  (8:32)  According  to  the  courts,  ihe  final  analysis  is  whether 
the  performance  appraisal  can,  in  fact,  measure  performance.  "It  must  be 
able  to  identify  or  distinguish  between  the  good,  average,  and  poor 
performer,  or  different  degrees  or  quality  of  performance  on  the  job." 
(15:17)  The  "rating  must  demonstrate  its  relatedness  to  the  particular  job 
being  rated  rather  than  to  generalized  factors  (appearance,  leadership, 
etc.)  that  may  or  may  not  have  a  significant  bearing  on  this  job."  (32:82) 
The  court  concluded  in  USA  v  City  of  Chicago  (1978)  that  "when  companies 
make  performance-based  decisions  on  the  basis  of  appraisals,  they  are  using 
the  appraisals  as  employment  tests;  thus,  they  must  be  based  upon 
identifiable  job-related  criteria."  (3:217) 

A  closely  related  issue  is  validity.  In  the  context  of  performance 
appraisals,  validity — a  statistical  concept — measures  the  degree  to  which 
the  performance  appraisal  actually  measures  what  it  is  intended  to  measure 
(e.g.,  job  performance  and  potential).  The  corollary  between 
job-re  I atedness  and  validity  is  simple.  The  more  related  the  appraisal  is 
to  the  job  being  evaluated,  the  more  valid  a  performance  appraisal  is.  If 
job-relatedness  of  the  performance  appraisal  is  low,  validity  is  low.  In 
Brito  v  Zia  Company  (1973)  the  court  ruled  against  the  company  and  its 
performance  appraisal  system  by  noting  that  the  company 


failed  to  validate  the  test  according  to  the  EEOC  guidelines 
because  it  failed  to  introduce  evidence  of  the  validity  of  its 
performance  evaluation  test  consisting  of  emperlcal  data 
demonstrating  that  the  test  was  significantly  correlated  with 
important  elements  of  work  behavior  relevant  to  the  jobs  for 
which  the  applicants  were  being  evaluated.  (22:111) 

In  summary,  the  courts  have  found  the  issues  of  job-rel atedness  and 
validity  as  important  "litmus  tests"  when  discussing  the  legality  of  a 
company's  performance  appraisal  system.  They  have  concluded  that 
performance  appraisal  systems  must  reflect  job-related  criteria  if  the 
criteria  are  to  be  considered  valid.  Furthermore,  the  courts  are  in 
agreement  that  validation  is  a  prerequisite  for  a  defensible  performance 
appraisal  system.  “When  the  Instrument  used  is  valid,  performance  appraisal 
is  legal.  Conversely,  when  the  instrument  is  not  valid,  or  even  when  the 
manager  fails  to  provide  evidence  of  its  validity,  performance  appraisal  is 
illegal."  (15:20) 


Closely  related  to  the  court's  requirement  for  performance  appraisals 
to  be  job-related  is  a  similar  requirement  for  the  appraisals  to  be  based  on 
a  job  analysis.  A  job  analysis  is  simply  a  method  companies  use  to 
thoroughly  understand  jobs.  Two  products  result  from  a  job  anal ysis--job 
descriptions  and  job  specifications.  A  job  description  is  the  tasks, 
duties,  and  responsibilities  Inherent  in  a  specific  job.  A  job 
specification,  though  similar,  expands  the  job  specification  by  listing  the 
unique  qualifications  necessary  for  job  success.  For  example,  a  job 
specification  typically  states  the  prerequisites  an  Individual  must  meet  to 
even  be  considered  for  a  job.  These  criteria  include  education,  experience, 
and  training.  Job  descriptions  and  job  specifications  combine  to  provide  an 
employer  the  most  relevant  and  necessary  aspects  of  a  job,  and  a  company 
determines  these  aspects  through  a  job  analysis. 

In  the  context  of  a  legal  performance  appraisal,  the  job  analysis  takes 
on  added  importance.  The  "appraisal  of  job  performance  must  be  based  upon 
an  analysis  of  job  requirements  as  reflected  in  performane  standards." 
(11:176)  Expanding  this,  once  a  company  is  finished  with  its  job  analysis, 
it  is  in  position  to  state  and  communicate  to  its  employees  the  performance 
standards  necessary  for  job  success.  Once  a  company  identifies  the 
performance  standards,  the  company  can  then  begin  to  constuct  and  ultimately 
achiinister  the  appropriate,  and  legal,  performance  appraisal  system. 

(11:176) 

In  a  landmark  court  decision  of  Albermarle  Paper  Company  v  Moody 
(1975),  the  court  agreed  that  "the  best  way  to  determine  whether  the 
appraisal  criteria  are  job-related  is  to  do  a  job  analysis."  (3:217)  The 
court  concluded  that  "one  of  the  reasons  International  [Albermarle!  Paper 
Company  was  unable  to  justify  certain  performance  appraisal  practices  was 
the  fact  that  the  rating  instrument  used  was  not  based  on  an  analysis  of  the 
job.  The  lack  of  a  job  analysis  made  it  impossible  to  show  the  ratings  were 


J ob  related.  .  .  (15:18)  In  Patterson  v  American  Tobacco  Company  (1976). 

the  court  reaffirmed  the  requirement  for  companies  to  conduct  a  thorough  job 
analysis.  The  court  concluded  that  the  company's  promotion  decision  was 
discriminatory,  and  consequently  illegal,  because  the  promotion  was  not 
based  on  "written  job  descriptions  and  objective  criteria  for  appointments 
[promotion] .“  (11:177) 

In  summary,  “the  courts  have  repeatedly  condemned  the  use  of 
performance-evaluation  instruments  that  have  not  been  developed  from  a 
systematic  analysis  of  the  job."  (8:33)  In  fact,  the  courts  more  recently 
have  expanded  this  view.  In  Carpenter  v  Stephen  F.  Austin  State  University 
(1983),  the  college  was  criticized  by  the  court  for  "using  obsolete  job 
descriptions  in  evaluating  its  employees.  Consequently,  periodic  job 
analysis  also  seems  to  be  important."  (8:33) 


A  third  area  that  courts  have  become  involved  in  with  respect  to 
performance  appraisals  is  in  the  nature  of  the  performance  criteria 
themselves.  There  are  basically  two  types  of  performance  appraisals  tnat 
distinguish  the  kinds  of  criteria  selected  to  measure  performance. 

Appraisal  forms  that  require  supervisors  to  evaluate  their  employees  in  such 
areas  as  leadership,  cooperation,  dependability,  leadership,  attitude  toward 
people,  and  loyalty  are  labeled  "subjective."  (3:219:  22:115)  Conversely, 
appraisals  that  require  employers  to  evaluate  their  employees  through 
"specifically  defined  behaviors"  (3:219)  are  labeled  "objective." 

Traditionally,  the  courts  have  discouraged  companies  from  using 
subjective  forms  of  performance  appraisals.  "The  Uniform  Guidelines  cleariy 
specify  (and  the  courts  have  generally  upheld)  that  employee  evaluation 
should  concentrate  on  job-specific  behaviors  rather  than  on  potentially 
relevant  traits,  abilities,  and  psychological  characteristics."  (8:33)  In 
Brito  v  21a  Company  (1973)  the  court  “ruled  that  such  aspects  as  appearance, 
ethical  habits,  and  loyalty  are  vague  and  subjective  and  may  not  have  any 
Impact  on  job  performance."  (8:33)  In  three  other  cases  (Albemarle  Paper 
Company  v  Moody  [1978],  Baxter  v  Savannah  Sugar  Refining  Corporation  [19741. 
and  Rowe  v  General  Motors  [19721),  the  courts  criticized  the  performance 
appraisal  systems  because  they  did  not  “produce  fair  or  accurate 
evaluations."  (3:219)  In  fact,  in  Rowe  v  General  Motors,  the  court  ruled 
that  the  promotion  system  General  Motors  used  was  in  violation  of  Title  VII. 
One  of  the  reasons  for  its  illegality  was  that  the  "standards  determined  to 
be  controlling  were  vague  and  subjective."  (26:17) 

The  courts  have,  however,  allowed  companies  to  use  more  subjective 
forms.  For  example,  in  Zell  v  United  States  (1979),  the  court  viewed 
favorably  the  subjective  performance  evaluations  because  it  was  supplemented 
by  more  “objective,  behavioral  measures  of  performance."  (8:33)  The  courts 
have  also  agreed  that  companies  may  find  it  extremely  difficult  to  use 
objective  criteria  in  evaluating  employees,  particularly  in  the  case  of 
professional  personnel.  (5:41)  "It  is  sometimes  necessary  to  identify 
specific  behavior  that  should  be  reinforced  and  rewarded."  (5:41) 


In  summary,  performance  appraisal  systems  may  be  Illegal  if  the  ratings 
are  based  on  an  evaluation  of  subjective  or  vague  factors.  (1:257)  Criteria 
that  are  based  upon  subjective  constructs,  upon  vague  or  subjective  factors, 
may  in  turn  not  meet  the  test  for  validity.  (32:82)  The  fundamental  reason 
for  the  courts'  stand  is  because  "vague  criteria  are  open  to  bias,  allowing 
different  interpretations  by  various  raters."  (15:18) 


COMMUNICATION 

Once  performance  standards  have  been  identified,  the  company  must 
communicate  those  standards  to  its  employees.  "The  courts  have  clearly 
stated  that  if  performance  evaluation  is  used.  .  .  ,  performance  criteria 
and  standards  must  be  communicated  to  the  employee."  (3:219) 

In  Rowe  v  General  Motors  (1972),  “the  court  ruled  that  one  of  the 
discriminatory  aspects  of  the  motor  company's  performance  appraisal  policy 
was  that  criteria  on  which  promotions  were  based  were  not  clearly 
communicated.  .  .  .“  (8:35)  Furthermore,  in  Zell  v  United  States  (1979), 
one  of  the  reasons  the  court  upheld  the  legality  of  the  performance 
appraisal  system  was  because  the  company  had  clearly  informed  its  employees 
on  the  standards  by  which  their  performance  would  be  measured.  (8:35) 

In  summary,  clear  communication  of  job  requirements  and  performance 
standards  may  often  make  forma!  legal  action  unnecessary.  Courts  have 
repeatedly  “reacted  negatively  to  performance  evaluation  systems  when 
standards  have  not  been  communicated  to  employees."  (8:35) 


RATER  TRAINING 

The  courts  have  clearly  favored  companies  that  spend  time  training  and 
indoctrinating  their  supervisors  in  how  to  properly  evaluate  their 
employees.  In  Carpenter  v  Stephen  F.  Austin  State  University  (1983),  the 
court  ordered  the  university  to  prepare  objective  and  thorough  written 
guidelines  to  assist  supervisors  in  their  evaluation  responsibilities. 

(8:36)  In  Rowe  v  General  Motors  (1972),  the  court  further  added  that 
without  written  evaluation  guidelines,  companies  would  be  unable  to  rate 
their  employees  on  a  constant  set  of  promotion  criteria.  (26:17) 
Unfortunately,  "supervisors  are  rarely  trained  to  collect  and  report 
behavior  in  an  objective,  constructive,  timely  and  useful  manner."  (32:82) 
The  result  is  that  such  a  system  breeds  biased  and  inconsistent  personnel 
decisions,  a  condition  the  courts  are  finding  difficult  to  defend.  The 
judges  are  now  frequently  asking  a  company  the  following  question:  Are 
supervisors  qualified  to  rate  someone  else's  performance?  (32:82)  "Poorly 
designed  [performance  appraisal!  systems  produce  poor  performance 
information  which  is  tough  to  defend.  But  equally  damaging  is  a  good  system 
that  is  neither  understood  nor  accepted  by  its  users  due  to  lack  of 
training."  (21:82) 
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RATER  CHARACTERISTICS 

A  final  area  courts  have  become  involved  in  is  the  company's  decision 
on  who  will  actually  conduct  the  evaluation.  Courts  have  ruled  that 
supervisors  must  have  regular  and  personal  contact  with  the  people  they 
evaluate,  or  the  system  itself  may  be  illegal.  ".  .  .  the  lack  of  daily 
supervision  results  in  a  lack  of  familiarity  with  the  performance  of  the 
subordinate.  Lack  of  familiarity  with  the  employee's  performance  introduces 
subjective  observations  and  results  in  an  illegal  appraisal."  (15:18) 

In  USA  v  City  of  Chicago  (1978),  the  courts  condemned  the  city's 
performance  appraisal  system  because  it  forced  supervisors  to  rate  employees 
when  supervisors  had  no  direct  knowledge  of  job  performance.  (25:92) 
Consistent  with  this  decision  was  the  one  made  by  the  courts  in  Brito  v  Zia 
(1973).  One  of  the  reasons  the  courts  declared  the  performance  appraisal 
illegal  was  because  “two  of  the  three  supervisory  evaluators  did  not  have 
daily  contact  with  the  employees  being  evaluated."  (26:17)  Finally,  in 
James  v  Stockholm  Valves  and  Fittings  Company  (1977),  the  courts  stated  that 
"unless  each  appraisal  dimensions  can  be  supported  by  objective,  observable 
evidence,  its  continued  use  may  well  be  prohibited."  (11:179) 

The  bottom-line  recommendation  the  courts  are  making  is  that  people  who 
conduct  evaluations  must  be  able  to  observe  the  behavior  they  are  rating. 
“Valid  ratings  cannot  be  made  by  someone.  .  .  unfamiliar  with  the  work  of 
the  ratee.  .  .  .“  (22:113) 


Chapter  Three 


LEGAL  GUIDELINES:  A  SUMMARY 

In  light  of  the  recent  involvement  of  the  courts  in  corporate 
performance  appraisals,  what  guidance  is  being  offered  to  the  private  sector 
to  make  its  performance  appraisals  more  defensible?  Management  consultant 
groups  and  academicians  have  published  a  wealth  of  suggestions  that 
companies  might  follow  to  ensure  a  legal  performance  appraisal  system.  In 
an  attempt  to  summarize  all  the  previously  mentioned  court  cases  within  the 
six  performance  appraisal  categories,  the  following  guidelines  briefly 
describe  the  most  relevant  suggestions  affecting  the  Air  Force's  Officer 
Effectiveness  Report. 


GUIDELINE  #1 

The  performance  appraisal  must  be  job-related.  The  job  characteristics 
and  performance  standards  contained  within  the  appraisal  should  depict  the 
essence  of  the  job.  The  characteristics  should  accurately  describe  the 
nature  of  the  work  and  the  standards  should  describe  the  level  of 
performance  necessary  for  job  success.  "The  performance  standards  (traits, 
behavior,  or  results)  are  based  upon  relevant  job  dimensions  that  are 
appropriate  to  the  nature  of  work  called  for  and  bear  upon  the  desired  job 
performance."  (21:82)  When  performance  appraisals  meet  this  test  for 
job-re latedness,  it  becomes  less  likely  that  criteria  and  dimensions  used  to 
describe  one  job  will  be  the  same  for  an  entirely  different  job.  (32:82) 


GUIDELINE  n 

The  content  of  the  performance  appraisal  should  be  based  on  a  thorough 
job  analysis.  This  will  preclude  employees  being  evaluated  on  “undefined, 
global  measures  of  job  performance.”  (11:76)  Rather,  a  job  analysis  will 
ensure  that  employees  are  evaluated  on  "clearly  defined  individual 
components  or  dimensions  of  job  performance."  (11:176)  In  addition, 
performance  appraisals  must  reflect  the  existing  job,  and  should  change  as 
the  job  changes.  This  requires  all  jobs  to  be  monitored  and  reviewed 
through  recurring  job  analyses  (8:36),  with  subsequent  changes  to  the 
performance  appraisal  when  appropriate. 


Performance  appraisals  must  be  valid.  When  validated,  each  performance 
appraisal  would  reflect  those  rating  dimensions  specific  to  the  job,  thereby 
ensuring  its  job-rel atedness.  The  use  of  "generic  designs  of  forms  for 
collecting  and  keeping  performance  data"  (32:82)  would  become  more  and  more 
difficult  to  defend  in  the  courts. 


Objective,  behavioral  1 y-based  job  dimensions  should  be  used  to  the 
fullest  extent  possible.  Ratings  on  construct  (subjective)  traits  (e.g.. 
dependability,  drive,  aptitude,  attitude,  initiative,  interpersonal 
communication  skills,  quality  of  work,  leadership,  cooperation,  stability) 
should  be  avoided.  (32:82:  22:115:  3:218) 


Subjective  appraisals  must  be  made  more  objective  to  the  fullest  extent 
possible.  A  number  of  methods  are  available  to  do  this.  First,  the 
performance  appraisal  should  be  formalized,  documented,  and  standardized 
company-wide,  (26:20)  “This  includes  statements  regarding  purposes  of 
appraisals,  mandatory  use  by  management,  complete  disclosure  of  the  program 
to  evaluation  and  due  process  procedures."  (21:81)  This  also  implies  that 
the  administration  and  scoring  of  the  performance  appraisal  system  are 
standardized  and  controlled.  (21:82)  Second,  the  raters  themselves  should 
be  trained  and  qualified  to  do  their  evaluative  job.  (22:118)  Third, 
subjective  ratings  should  be  only  one  input  to  the  entire  evaluation 
process,  particularly  to  subsequent  personnel  actions,  such  as  promotions. 
(4:237:  26:21)  Fourth,  "behavioral  documentation  should  be  required  for 
extreme  ratings."  (3:218) 


An  employee  evaluation  should  be  conducted  by  multiple,  independent 
appraisers  with  personal  contact  with  the  ratee.  (3:218;  26:21)  Each 
appraiser  should  have  continual  and  personal  contact  with  the  employee,  and 
be  afforded  ample  opportunity  to  observe  his  or  her  performance.  (3:218: 
4:237;  25:94)  Such  action  will  prevent  one  supervisor  from  acting  aione. 
thus  possibly  controlling  unilaterally  the  employee's  career.  (25:94) 


Supervisors  must  be  trained  for  their  evaluative  responsibilities.  A 
training  program  should  Involve  (1)  written  instructions  for  completing  the 
performance  appraisal  (1:257-258);  (2)  laws  affecting  performance  appraisals 
and  discrimination  (3:218):  (3)  what  the  intended  uses  of  the  appraisal  are 
(4:237):  (4)  an  orientation  of  what  the  supervisory  responsiDi i i ties  are 


(4:237);  and  (5)  the  skills  required  of  evaluators.  These  skills  include 
the  proper  motivation  to  evaluate,  "a  clear  understanding  of  the  performance 
appraisal  instrument,  the  words  and  weights  used  in  it,  the  rating  errors 
that  can  be  made,  knowledge  of  what  behavior  should  or  should  not  be  rated, 
and  the  awareness  of  one's  own  prejudice  and  how  they  affect  the  rating." 
(15:18-19) 


GUIDELINE  #8 

The  results  of  the  appraisal  should  be  reviewed  by  the  supervisor  to 
the  employee  throughout  the  entire  evaluation  process.  This  will  allow  the 
employee  to  air  any  appraisal  disagreements.  (3:218)  Included  in  this 
review  should  be  a  formalized  appeal  process  if  an  employee  disagrees  with 
an  evaluation.  (4:237;  11:177) 


GUIDELINE  49 

The  performance  appraisal  system  should  be  communicated  to  and 
understood  by  the  employees.  (3:218)  This  should  include  an  explanation  of 
the  job  dimensions  and  standards,  how  the  system  will  be  administered,  and 
what  impact  it  will  have  on  future  personnel  actions. 


GUIDELINE  #10 

Personnel  decisions  based  on  the  performance  appraisal  must  not  be 
discriminatory.  Accordingly,  “an  organization  should  keep  accurate  records 
of  who  is  eligible  for  and  interested  in  the  promoted  position.  It  should 
also  initiate  a  strong  EEO  [Equal  Employment  Opportunity]  posture,  which 
would  include  the  implementation  of  an  affirmative  action  program."  (22:118) 


Chapter  Four 


THE  OER  AND  THE  COURTS 

Before  comparing  the  Air  Force's  performance  appraisal  system  witn  the 
ten  guidelines  summarized  in  Chapter  3.  a  quick  look  at  USAF  source 
documents  on  officer  evaluations  is  appropriate.  AFR  36-10,  Officer 
Evaluations,  is  the  definitive,  officially-sponsored  guidance  package  on  the 
OER.  The  regulation  "estaolishes  Air  Force  policy  regarding  officer 
evaluations.  It  tells  how  to  prepare,  suDmit.  and  control  AF  Form  707. 
Officer  Effectiveness  Report.  .  (35:1)  Consequently,  tne  major  tnrust  of 

the  regulation  is  administrative  in  nature,  and  therefore  offers  iittie 
insight  into  the  degree  the  Air  Force  meets  or  fails  to  meet  court- i ssueo 
guidelines.  MAJCOM  guidance  is  less  helpful  in  determining  the  OER  s 
legality.  It  must  De  emphasized  that  "all  supplemental  directives  tnat 
change  basic  policies  or  merely  reiterate  or  duplicate  the  text  of  this 
regulation  [AFR  36- 1 C 3  are  prohibited."  (35:1)  MAJCOM  supplements  to  AFR 
36-10  therefore  only  provide  guidelines  intended  to  streamline 
administrative  support  specific  to  the  MAJCOM.  In  addition  to  these 
supplements,  each  MAJCOM  has  issued  a  guide  for  rating  officials,  all  quite 
similar  in  nature  and  content.  These  pamphlets  provide  communicative 
techniques  that  allow  raters  to  become  more  effective  in  tneir  written 
words.  Essentially,  they  advise  the  rater  on  how  to  write  an  OER  tnat  is 
"universally  understandable,  specific,  and  concise." 

Given  the  lack  of  official  Air  Force  guidance  on  the  OER,  this  paper 
assumes  that  there  is  no  formal  Air  Force  guidance  unless  specifically 
mentioned  in  AFR  36-10.  If  local  units  have  adopted  programs  that 
ultimately  make  the  OER  more  " 1 egal -proof . "  they  have  done  so  at  tneir  own 
initiative  and  without  formal  Air  Force  direction. 

This  chapter  examines  each  of  the  ten  guidelines  mentioned  in  Chapter  3 
to  the  Air  Force  OER  system.  It  will  describe  the  extent  to  which  the  Air 
Force  meets  or  fails  to  meet  these  guidelines.  Following  this  comparison, 
it  will  conclude  with  possible  solutions  the  Air  Force  might  adopt  to  Detter 
comply  with  the  court-issued  guidelines. 


GUIDELINES  #1  AND  »2 

The  first  two  guidelines  are  inextricably  interrelated  with  each  other. 
The  primary  method  to  ensure  job-re  1 atedness  (guideline  #1)  is  to  conduct  a 
job  analysis  (guideline  #2).  Both  issues  impact  only  on  the  front  side  of 
the  OER,  specfically  on  the  ten  performance  factors  (see  appendix  A).  In 


the  context  of  the  OER,  the  courts  are  advising  that  these  performance 
factors  be  job-related,  and  the  selection  of  the  factors  be  based  on  a  job 
analysis. 

Are  the  performance  factors  job-related?  In  a  telephone  interview,  the 
Office  of  Primary  Responsibility  (OPR)  to  AFR  36-10,  HQ  AFMPC/DPMYO, 
admitted  that  to  the  best  of  its  knowledge,  no  historical  record  is 
available  that  describes  how  the  current  performance  factors  were  selected. 
Lacking  this  information  and  due  to  the  generic  form  that  evaluates  all 
officers  alike,  this  author  concludes  that  the  OER  is  not  job-related. 

As  previously  mentioned,  the  courts  are  beginning  to  demand  that 
performance  appraisals  be  "tailored  to  the  unique  nature  and  expectations  of 
each  job."  (17:54)  “Managers  should  beware  standardized  review  forms. 

These  forms  tend  to  focus  on  behavioral  traits  and  personality  factors  and 
may  be  illegal  because  they  may  not  be  pertinent  to  the  job  being  reviewed." 
(17:54)  In  adopting  one  standardized  form  to  appraise  every  officer  job 
below  the  rank  of  colonel,  the  Air  Force  seemingly  violates  this  job-related 
advice.  There  are  over  250  Air  Force  Specialty  Codes  (AFSC)  listed  in  AFR 
36-1,  excluding  prefix  or  suffix  combinations  or  entry  level  AFSCs  for  each 
career  field.  (34: — )  It's  difficult  to  believe  that  the  ten  performance 
factors  are  uniquely  tailored  and  therefore  perfectly  matched  to  each  and 
every  Air  Force  officer  AFSC. 

The  solution  to  this  over-reliance  on  one  form  for  all  officers  lies 
with  the  job  analysis.  Ironically,  the  Air  Force  has  already  conducted  a 
rudimentary  job  analysis;  this  effort  is  reflected  in  AFR  36-1.  This 
regulation  provides  a  brief  job  description  and  the  necessary  qualifications 
(job  specification)  for  each  primary  AFSC.  As  previously  mentioned,  these 
two  descriptors  are  the  end-products  of  a  job  analysis.  Unfortunately,  it 
is  evident  that  the  Air  Force  has  never  attempted  to  tie  AFR  36-1  to  the 
OER.  Even  though  technology  has  exploded  over  the  last  decade,  dramatically 
changing  some  jobs  and  creating  others,  the  OER's  performance  factors  have 
remained  unchanged. 

There  are  several  things  the  Air  Force  can  do  to  make  its  OER  more 
job-related.  By  directly  tying  AFR  36-1  to  the  OER,  the  Air  Force  can  make 
the  OER  more  relevant;  that  is,  it  can  ensure  a  clear  linkage  "between  the 
critical  job  elements  identified  through  a  job  analysis  and  the  dimensions 
to  be  rated  on  an  appraisal  instrument."  (10:70)  In  addition,  AFR  36-1 
could  be  supplemented  by  conducting  observations  and/or  quest ionaires  with 
people  actually  working  within  each  AFSC.  This  additional  job  data  would 
strengthen  AFR  36-1  and,  in  turn,  make  the  evolving  OER  even  more  relevant. 

The  result  of  relating  AFR  36-1  to  the  OER  may  be  multiple  designs  of 
the  OER  that  better  distinguish  the  numerous  career  fields  in  the  Air  Force. 
As  more  and  more  emphasis  is  placed  on  AFR  36-1,  the  Air  Force  should  ensure 
its  accuracy  by  conducting  periodic  reviews  of  each  AFSC  and  making 
appropriate  changes  to  AFR  36-10.  If  AFR  36-1  substantially  changes,  the 
corresponding  OER  should  change.  At  all  times  the  Air  Force  should  avoid 
using  obsolete  job  descriptions  and  specifications  as  inputs  to  its  OER. 


The  OER  is  not  valid  for  two  reasons.  First,  since  the  performance 
factors  are  minimally  related  to  the  job  being  appraised,  the  validity  is 
correspondingly  low.  Second,  given  the  high  degree  of  inflation  tnac 
currently  exists  within  the  OER  system,  even  if  the  factors  were 
job-related,  it  is  doubtful  the  appraisal  would  be  any  more  valid.  "The 
presence  of  rating  errors  [inflation]  serves  to  minimize  validity.”  (22:113) 

Interestingly,  AFR  36-10  specifically  forbids  inflation.  Commanders 
are  reminded  that  “inflationary  tendencies  are  not  permitted.”  (35:11) 
Furthermore,  AFR  36-10  states  that  "evaluators  are  responsible  for  preparing 
fair,  objective  evaluations  that  accurately  differentiate  among  officers.” 
(35:11)  Obviously  this  AF  regulation  has  done  little  to  discourage 
inflation.  One  plausible  reason  for  this  high  inflation  Is  the  multiple 
purposes  the  OER  has  within  the  Air  Force  personnel  system.  (9:50)  One 
study  suggested  that  “the  biggest  reason  for  performance  appraisal  failure 
is  too  many  uses  and  conflicting  objectives.  The.  .  .  problem  is  the 
multiplicity  of  the  appraisal  form  or  actual  use  of  the  data  for  performance 
feedbacK.  compensation,  identifying  potential,  making  promotional  decisions, 
personnel  research,  development  and  training,  etc.”  (24:70) 

Solutions  that  are  available  to  ensure  a  valid  performance  appraisal 
therefore  center  around  two  themes-- job-re  I atedness  and  inflation.  This 
paper  has  already  provided  possible  solutions  to  solve  the  job-related 
issue.  Solutions  in  the  literature  that  prevent  inflation  can  be  broken 
down  into  three  general  categories.  The  first  category  attempts  to  demand 
objectivity  from  the  rater.  Managers  believe  that  if  the  rater  is  evaluated 
in  his  or  her  rating  skills  ahd  tendencies,  rater  inflation  will  be  reduced. 
(13:77-79;  18:640-649:  30:5)  This  belief  is  reflected  in  the  Army  s  OER. 
wherein  the  Army  maintains  a  rating  profile  of  each  endorsing  official. 

This  rating  profile  reflects  the  rating  history  of  the  senior  rater  with 
respect  to  potential  (e.g.,  the  percentage  of  "Is",  "2s",  "3s"  he  or  she  nas 
given  over  a  period  of  time).  The  second  category  attempts  to  enrich  the 
performance  evaluation  by  broadening  the  rater  base  to  include  peer  and 
subordinate  ratings.  One  study  at  the  Air  War  College  discussed  this  theme, 
concluding  that  peer  and  subordinate  ratings  are  "valid,  reliable,  and 
acceptable"  (29:20)  and  would  be  an  effective  supplement  to  the  Air  Force  s 
current  OER.  (29:20)  The  final  category  attempts  to  diffuse  the  multiple 
purposes  an  organization  often  gives  to  appraisals.  Appraisals  are 
traditionally  used  to  evaluate  current  performance  as  a  coaching  and 
developmental  tool,  and  to  rate  potential  for  promotion.  (6:29-30)  Authors 
agree  that  these  two  Important  tasks  should  not  be  contained  within  the  same 
performance  appraisal.  (6:30;  28:60-62;  33:130-131;  19:56)  All  conclude 
that  performance  appraisal  systems  that  separate  performance  evaluations 
from  potential  assessments  lead  to  more  valid,  less  inflated  rating  systems. 


GUIDELINES  #4  AND  #5 


While  the  courts  conclude  that  objective  job  dimensions  are  preferred 
over  subjective  dimensions,  they  recognize  that  companies  often  cannot  avoid 


using  subjective  criteria.  As  previously  mentioned,  particularly  within  a 
professional  or  managerial  setting,  “it  is  often  necessary  to  identify 
specific,  subjective  behavior  that  should  be  reinforced  and  rewarded." 

(5:41)  This  legal  sentiment  fortunately  favors  the  Air  Force.  If  asked, 
the  courts  would  determine  the  majority  of  the  OER's  performance  factors  to 
be  subjective  in  nature.  Consequently,  the  Air  Force  must  make  its 
subjective  form  as  objective  as  possible  by  following  the  four  guiaelines  or 
safeguards  listed  on  page  nine. 

First,  the  performance  appraisal  should  be  formalized,  documented,  and 
standardized  company-wide.  The  Air  Force  has  effectively  achieved  this 
requirement.  AFR  36-10  is  the  sole  authority  on  officer  evaluations.  The 
OER  program  is  formalized  and  documented  within  this  regulation,  and  its  use 
is  standardized  through  mandatory  compliance  Air  Force  wide.  The 
administration  and  scoring  of  the  OER  is  also  standardized  and  controlled, 
as  evidenced  by  the  central  promotion  system  currently  practiced  by  MFC. 
Furthermore,  in  an  attempt  to  make  the  subjective  performance  factors  more 
objective,  AFR  36-10  provides  performance  standards  relative  to  each  factor 
in  the  five  rating  categories:  far  below  standard,  below  standard,  meets 
standard,  above  standard,  well  above  standard.  (35:47-48)  Each  standara  is 
further  described  in  as  objective  and  observable  behavior  as  possible  (see 
appendix  B). 

Second,  the  raters  should  be  trained  to  do  their  evaluative  joo.  The 
Air  Force  substantially  fail3  this  requirement.  While  formal  and  informal 
guidance  is  available  through  MAJCOM  sponsored  pamphlets,  there  exists  no 
program  that  ensures  every  rater  is  trained  for  the  job  he  or  she  is 
required  to  perform. 

Third,  subjective  ratings  should  be  only  one  input  to  the  entire 
evaluation  process.  This  is  certainly  true  within  the  Air  Force.  While 
OERs  are  certainly  the  most  important  criteria  for  promotion,  others  are 
considered  by  the  promotion  board.  For  example,  in  following  the  "total 
person"  concept,  board  members  look  not  only  at  the  OER,  but  at  the 
officer's  education  (professional  and  civilian),  experience  base,  and 
assigned  history,  just  to  name  a  few. 

Finally,  documentation  must  be  included  for  extreme  ratings.  The  Air 
Force  achieves  this  requirement  by  stipulating  that  specific  examples  of 
performance  be  recorded  for  each  performance  factor,  regardless  of  the 
rating.  (35:42) 

The  only  suggestion  for  improvement  in  this  area  naturally  falls  under 
rater  training.  This  requirement  is  mentioned  under  an  entirely  separate 
guideline,  and  wi i 1  be  discussed  later. 


GUIDELINE  H 


The  Air  Force  receives  both  passing  and  failing  marks  under  this 
guideline  of  multiple,  independent,  and  knowledgeable  appraisers.  The  Air 
Force  demands  at  least  two  raters  for  each  OER,  with  the  norm  being  three. 


This  requirement  ensures  that  one  rater  cannot  act  alone  and  thereby  control 
an  officer's  career.  Even  though  there  are  multiple  appraisers,  it  does  not 
imply  that  their  appraisals  are  independent  of  each  other.  It  is  the  Air 
Force  practice  to  have  the  primary  rater  provide  suggested  additional  rater 
and  indorser  comments  for  these  two  individuals.  To  the  degree  that  these 
officials  completely  accept  or  reject  the  suggested  remarks,  independence 
may  or  may  not  be  achieved.  Finally,  the  Air  Force  has  long  recognized  tne 
importance  of  all  rating  officials  having  personal  knowledge  of  the  ratee  s 
performance.  In  fact,  in  a  1985  change  to  AFR  36-10,  the  Air  Force 
stipulates  that  the  additional  rater  be  the  rater's  rater,  or  "someone  lower 
in  the  rating  chain  when  deemed  appropriate."  (36:1)  Furthermore,  AFR  36-10 
suggests  that  the  indorser  be  the  additional  rater's  rater,  with  a  reminder 
that  “deviations  from  the  normal  chain  of  supervision  must  be  judicious." 
(36:1)  It  is  clear  that  the  Air  Force  is  doing  its  best  to  meet  the 
requirement  that  all  raters  be  personally  aware  of  the  ratee's  performance. 
Unfortunately,  given  the  high  percentage  of  general  officer  indorsements 
throughout  all  ranks,  more  can  be  done.  Two  suggestions  follow. 

The  1987  ACSC  OER  Working  Group  offered  a  practical  solution  to  the 
above  problem.  In  advocating  a  strict  compliance  with  the  rating  chain  (as 
determined  by  the  AF  Form  2095).  the  board  recommended  that  colonels  oe  the 
final  indorser  for  majors  and  below.  (38:1)  This  obviously  minimizes 
general  officer  indorsements  at  the  junior  ranks,  and  more  directly  forces 
the  rating  officials  to  be  at  the  lowest  possible  levels  in  the  chain  of 
supervision.  The  second  suggestion  was  discussed  under  validity  and  could 
be  used  to  keep  the  level  of  personal  contact  with  the  ratee  as  high  as 
possible.  By  adopting  peer  and  subordinate  ratings,  not  only  oo  you 
increase  the  number  of  independent  appraisers,  but  you  allow  those 
individuals  that  are  perhaps  closest  to  the  ratee  to  judge  the  real  worth  of 
the  ratee's  performance. 


The  courts  are  very  clear — companies  should  provide  rater  training  to 
all  of  their  supervisors.  The  Air  Force  falls  exceedingly  short  in  meeting 
this  guideline.  AFR  36-10  provides  minimal  guidance  on  the  commanders 
responsibility  in  training  their  evaluators.  The  commanders'  general 
obligation  is  the  ensure  that  their  “evaluators  and  personnel  officers  have 
been  Instructed  on  the  contents  of  this  regulation  [AFR  36-101  so  that 
officer  evaluation  reports  are  accurate,  impartial,  and  accomplished  on 
time."  (35:11)  Beyond  this  statement,  there  is  no  official  guidance  tor  any 
kind  of  training  program.  Air  Comnand  and  Staff  College  has  an  entire  block 
of  instruction  set  aside  for  the  OER.  Unfortunately,  only  a  small 
percentage  of  the  entire  officer  population  attends  the  school.  Each  MAJCOM 
issues  a  guidance  package  on  how  to  write  OERs.  While  certainly  in  a  step 
in  the  right  direction,  those  packages  are  still  not  formulated  into  a 
formal  training  program.  With  respect  to  SAC's  guide  to  the  OER,  SACP  36-1. 
the  MAJCOM  leaves  it  up  to  the  discretion  of  local  CBPOs  or  MPOs  on  how  the 
pamphlet  will  be  used  in  the  units.  As  a  prior  SAC  evaluator,  this  author 
came  upon  SACP  36-1  only  by  chance,  and  has  never  read  AFR  36-10. 


The  Air  Force  solution  to  this  training  deficiency  is  simple.  The  Air 
Force  should  adopt  a  formalized  training  program  that  is  implemented  by  the 
local  CBPO  or  MPO.  Ancillary  to  this  program  would  be  an  Air  Force 
sponsored  OER  information  package  similar  to  those  provided  by  the  MAJCOM 
that  supplements  AFR  36-10.  Each  evaluator  would  be  required  to  attend  a 
primary  course  on  evaluative  skills,  with  possible  periodic  refresher 
courses.  Basically,  the  CBPO  or  MPO  sponsored  training  workshop  would 
introduce  the  necessary  skills  required  to  properly  evaluate  officers. 
Following  is  a  sampling  of  the  kinds  of  orientation  and  skills  the  workshop 
would  include: 

-  Clear  understanding  of  the  OER 

-  Written  instructions  for  completing  the  OER  (e.g.,  SACP  36-1) 

-  Laws  affecting  OERs  and  discrimination 

-  Intended  uses  of  the  OER 

-  Orientation  of  what  the  supervisory  responsibilities  are 

-  Proper  motivation  to  evaluate 

-  Rating  errors  that  can  be  made  (e.g.,  rater  bias) 

-  Knowledge  of  what  behavior  should  or  should  not  be  rated 

-  Evaluator  skills,  such  as  setting  objectives,  providing  feedback, 
conducting  interviews,  listening,  and  communication  skills  (31:39) 


The  courts  simply  suggest  the  appraisal  be  reviewed  formally  between 
the  rater  and  ratee,  and  a  company  have  a  formalized  appeal  process.  AFR 
36-10  places  little  emphasis  on  the  OER's  use  for  feedback,  or  as  a 
counseling  tool.  Only  one  paragraph  is  alloted  to  counseling  and  its 
importance.  It  states  that  the  OER  should  not  be  used  as  a  "counseling 
device,"  (35:6)  “You  should  counsel  as  the  need  arises.  Counseling  is  a 
day-to-day  process."  (35:6)  More  formalized,  periodic  counseling,  if 
appropriate,  may  be  conducted  twice  a  year.  (35:6)  Despite  these  words  in 
AFR  36-10,  and  unless  this  author's  personal  experience  is  the  exception, 
little  effort  is  made  Air  Force  wide  to  use  the  OER  as  a  vehicle  to  provide 
legitimate  feedback  to  the  ratee.  On  the  otherhand,  the  Air  Force  does  have 
a  formal  appeal  process  within  the  evaluation  process.  AFR  31-3  provides 
the  formal  appeal  process  that  officers  can  use  to  question  and  appeal 
perceived  unfair  ratings. 


AFR  36-10  needs  to  better  establish  the  requirement  for  counseling 
throughout  the  evaluation  process.  It  must  take  out  such  expressions  "as 
the  need  arises"  and  “if  deemed  appropriate"  (35:6)  and  replace  them  with  a 
more  formalized  and  enforced  counseling  and  feedback  program.  Again,  the 
1987  ACSC  OER  Working  Group  offered  a  practical  solution.  In  their 
recommendation,  a  separate  Input  Form  would  be  used  to  formally  document 
three  counseling  sessions  with  the  rating  cycle.  This  form  would  "neither 
become  a  permanent  part  of  the  officer's  record,  nor  will  it  accompany  the 
OER  up  the  chain  as  the  OER  is  finalized;  therefore,  most  of  the 
inflationary  pressure  is  removed  from  the  counseling  form,  allowing  it  to 
reflect  an  accurate  picture  of  the  ratee's  performance."  (38:7) 


V 


Further  research  offers  a  number  of  techniques  designea  to  improve  cne 
ievei  ana  content  of  feedback  between  the  rater  ana  ratee.  Performance 
appraisal  interviewing  ana  review  techniques  (27:74-79:  l 2 : 1 5- 1 7 :  l 6 : 11-16: 
9:28-36)  offer  the  rater  with  the  skills  necessary  in  his  feedback 
responsibilities.  In  addition,  a  new  technique  of  sel t-appraisai  (14:22: 
10:73-74:  6:31)  allows  the  ratee  to  contribute  to  his  or  ner  own  counsel ina 
sessions  in  a  unique,  non-tradi t i ona i  way. 


The  ratee  should  have  a  clear  understanding  of  the  performance 
appraisal  system.  Consequently,  the  courts  place  strong  emphasis  on  tne 
communicative  cnannels  available  to  employees  with  respect  to  evaluations. 
The  Air  Force  relies  almost  totally  on  AFR  36-10  as  tne  sole  formal 
communicative  tool  for  OERs.  Unfortunately,  there  is  no  statement  witnin 
tne  regulation  that  formally  requires  all  Air  Force  personnel,  raters  ana 
ratees,  to  read  the  document.  Therefore,  the  Air  Force  relies  equally  as 
much  on  an  effective  informal  channe 1 --word  of  mouth.  Having  never  read  AFR 
36-10,  this  author  came  to  understand  the  OER,  with  ah  its  perfections  arid' 
imperfections,  through  the  eyes  of  feliow  officers.  This  author  is 
confident  his  experience  is  similar  to  the  majority  of  Air  Force  officers. 

To  strengthen  this  particular  aspect,  several  suggestions  are  offered. 

First,  the  Air  Force  should  require  all  commissioning  sources  to 
indoctrinate  all  future  officers  in  the  OER  and  AFR  36-10.  For  examDie.  tne 
US  Air  Force  Academy  has  an  entire  block  of  instruction  during  the  cadet  s 
finai  year  on  officer  evaluations.  As  part  of  the  caaet's  formal 
professional  military  training,  this  block  introduces  cadets  to  the  OER  ana 
how  it  is  used  in  the  Air  Force.  Second,  officers,  as  part  of  their 
in-processing  at  their  first  duty  assignment  after  receiving  their 
commission,  should  be  briefed  by  the  local  CBPO  or  MPO  on  OERs.  Tne  agenda 
snouid  include  the  following: 

-  Purpose  of  the  OER  (7:37) 

--  Promotion  assessment 
--  Career  development 
--  Sk i i 1 s/ know! edge  data 
--  Vehicle  for  feedback 

-  Administration  of  the  OER 

-  Performance  standards  on  which  they  are  to  oe  evaluated 

-  Importance  to  any  current  and  future  personnel  actions 

Finally,  officers  should  take  a  refresher  course  on  officer 
evaluations,  particularly  during  times  when  the  OER  or  its  administration 
substantively  changes.  One  study  suggested  that  employees  be  part  of  tne 
formal  training  program  designed  for  the  supervisor.  This  "highly 
participative  approach  will  reduce  the  dissonance  that  can  resuit  from 
appraisal  training,  increase  the  desire  to  find  a  meaningful  use  of  tne 
system,  ana  clarify  what  expectations  each  layer  of  management  nas  of  tne 


layer  above  and  the  layer  below."  <20:52)  This  could  be  easily  implemented 
within  the  Air  Force,  The  local  CBPQ  or  MPO  could  identify  overlapping 
areas  of  interest  and/or  importance  within  the  rater  training  and  the 
ratee's  OER  refresher  course,  and  have  both  groups  attend  during  those 
times. 


In  every  decision  impacting  performance  appraisals,  the  courts  are 
fundamentally  demanding  that  companies  not  discriminate  in  their  personnel 
decisions.  As  an  aggressive  spokesperson  for  equal  opportunity  and 
affirmative  action,  the  Air  Force  is  unequivocably  successful  in  meeting 
this  guideline.  As  just  one  example,  the  Air  Force  monitors  promotion  rates 
of  every  category  in  the  Air  Force  population  to  ensure  minority 
representation  goals  are  achieved  over  time. 


Chapter  Four 


SUMMARY  AND  CONCLUSION 

The  number  of  civil  court  cases  challenging  the  legality  of  corporate 
performance  appraisals  is  dramatically  increasing.  With  each  new  decision, 
the  courts  are  generously  providing  companies  free  advice  on  how  to 
construct  and  implement  their  performance  appraisal  systems.  The  last 
several  years  of  court  battles  have  been  summarized  into  ten  operating 
guidelines  that  particularly  impact  the  Air  Force  Officer  Effectiveness 
Report.  The  Air  Force  receives  some  high  marks  and  some  low  marks  in  its 
unintentional  attempt  to  make  its  OER  legal.  Using  the  following  grading 
criteria,  a  summary  follows  that  depicts  the  extent  the  Air  Force  complies 
with  each  legal  guideline,  and  suggestions,  when  appropriate,  on  possible 
ways  to  improve  its  "grade." 
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"A" 
“  B" 
“C" 
"D" 

II  pu 


Total  compliance;  no  changes  required 
Satisfactory  compliance;  minor  changes  recommended 
Satisfactory  compliance;  major  changes  highly  recommended 
Marginal  compliance;  substantive  changes  required 
Total  uncompl lance;  complete  overhaul  required 


SUMMARY 


Guideline  H  ( Job-relatedness) ■  D 
Suggestions: 


-  Integrate  AFR  36-1  into  OER 

-  Multiple  OER  designs 


Guideline  #2  (Job  analysis):  B 
Suggestions: 


Supplement  AFR  36-1  with  observations  and  quest ionaires 
Conduct  periodic  AFR  36-1  revisions  and  updates 
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Guideline  S3  (validity):  F 


► 


Suggestions: 

-  Incorporate  suggestions  to  improve  joo-re I atedness 

-  “Rate-the-rater"  program  to  ensure  rater  oojectivity 

-  Broaden  rater  case  (peer  and  suDoroinate  ratings) 

-  Decouple  evaluation  from  potential  assessment 


a 


A 


.1 


Guidelines  S4  and  IS  (ob.iectlve/sub.iectlve  criteria):  C 
Suggestions: 


> 


-  Rater  training 


Guideline  s6  (multiple,  independent,  knowledgeable  appraisers):  B 
Suggestions: 

-  Strict  compliance  with  rater  chain 

-  Limit  general  officer  indorsements 

-  Peer  and  subordinate  ratings 

Guideline  #7  (rater  training):  p 


» 


> 


3 


a 


Suggestions: 

-  Adopt  formal  Air  Force  wide  training  program 

-  Distribute  formal  AF  pamphlet  (similar  to  SACR  36-1)  to  aii 
evaluators  on  how  to  use  and  write  the  OER 


Guideline  s8  (performance  review;:  G 

Suggestions: 

-  Formalized,  documented  counseling  program 

-  Performance  interviews  and  review  sessions 

-  Sei f -appraisal  program 


Guideline  S9  (ratee  communication):  c 
Suggestions: 

-  Orientation  conducted  at  commissioning  sources 

-  Refresher  training/orientation  conducted  at  local  CBPOs  or 
MPQs 


> 

si 


Si 


► 
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Suggestions: 


-  No  changes  recommended 


CONCLUSION 

From  the  above  review,  it  is  clear  that  much  work  is  required  for  the 
Air  Force  to  better  defend  its  officer  evaluation  from  a  legal  perspective. 
Some  suggestions  would  radically  change  the  nature  and  design  of  the  report 
itself.  Other  suggestions  could  be  implemented  regardless  of  the  OER 
design.  Compliance,  while  ensuring  a  more  defensible  system  if  ever 
challenged  by  the  courts,  would  have  a  more  immediate  effect.  The  OER  would 
simply  become  a  much  more  effective,  relevant,  and  responsive  management 
tool  for  a  variety  of  personnel  activities  (e.g.,  officer  evaluation,  career 
development,  promotion,  training,  and  performance  feedback).  In  any  event, 
there  is  much  talk  within  the  private  and  public  sectors  on  corporate 
responsibility  vis-a-vis  performance  appraisal  systems.  The  courts  are 
talking:  the  Air  Force  should  begin  listening. 
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•  Effectively  balances  resources 

•  I  i  usS'cd  products  aie  corcj-.enUy 
su1,  mitcd  on  time 

4  •  L  as  .  uti.ium  maieruu  with  gool  ies-!ts 

»  L'-_it’.shcs  cnntrtls  *o  ensure  that 
r.j  : power  and  nsat-.ul  aie  actour.ied  f  >t 
• ' i  ^.r-nserved 

•  D.-.eln  s  and  uses  c-st-cttce'ixe  methods 


5  •  Accepts  rrspomhtLtv  of  assigned  tastt 

•  Consistently  display  a  iritiauxe 

•  Commands  respect  of  subordinates 

•  It  tau  and  comment  m  deiiirg  with 
subordinates 


6  •  •  if  *  i‘ le  an-J  open  *^  .*••*  ideas 

•  ^tiirgly  seeks  assistance  in  ddfieult 

v 

•  r--  .ids*  reliable  d?s  is:  -ns  under  p  '•sure 

•  C  n  sunnily  dijpljsa  cairn  and  : -n n : Jed 

behavior 

1  •  direct  and  understandable  responses 

•->  questions 

•  On cS  brtefingi  which  are  '•rgimred  and  v*-eU 
presented 


8.  •  V.  'j’u’l  is  clear  an  1  .  "Crse 

*  Written  instruction  and  reports  a;*  readiiv 

•  ’«* ri*!«; n  c-n~u-  >’i  ns  are  c  ■- *j “end. 
*  ell  orrii  :;-d  and  g: :  ">  n  me  all'*  :  "re  t 


9  •  h  jjiajr  jf  and  U  jo*i  \ :  r  f  *rr  p~o-;« 

-  J  i.  t  rtti'-ci 

•  Pc-:-jns  urrrm  *.n  develop-e  n  is  and 
procedures 

•  C.vjfvitn  fuMy  with  re*  ideal  »r d  policies 

•  Volunteers  fn  additional  dunes  and 
promotes  pjrti::pat:-  s 

•  Brim*,  behavior  a^d  jp  mirg  create  a 
f  d  •mptettum 

•  I  "tri  e  in  •o.*'  ng  with  peers  and 
subord. rates 


ABOVE  STANDARD  IAII  Mamal 

•  f  sir  vie  i  veen  iniijthi  and  the  ability  |  .  e»<  l»e  tt  • 
on  •  pra*  '*  al  s«-!uu><ns 

•  krepi  inf-TineJ  of  important  developments  in  • 
related  fields 

•  (an  handle  difficult  iiluationi  effccuvely 

•  Broad  knowledge  of  related  millions  • 

•  Rarely  requires  guidance  or  aumance 


•  An  exceptionally  sound,  topical  thinker 

•  Does  not  hesitate  to  make  req-.iucd  decisions 

•  Decisions  are  consistently  coneci 

•  Opinions  and  judgment  are  often  solicited  by 
others 


•  Plans  bevond  requirements  of  present  job 

•  Plans  coincide  with  related  activities 

•  Is  flexible  and  able  to  adjust  priorities 

•  F  requently  called  on  to  otganize  complex  tasks 

•  f  xcellem  ■  •  suits  accomplished  at  minimum  cost 

•  Conustenuy  suggests  methods  of  ccnserving 
resources 

•  Skillfully  uses  cost-ef  fee  ;ne  ness  studies 


•  Demonstrates  a  high  degree  of  imtiauvt  and 
acceptance  of  responsibility 

•  Display  s  ex  ;cp  jcnaJ  skill  in  directing  others 

•  Promotes  enthusiasm  fc,  interest  ard  sincerity 

•  Acknowledged  leader  among  his  or  hei  peers 


•  PeaJJy  adapts  to  riuctuauoni  and  changing 
pr>;  ritrrs 

•  C  yfissstentlv  perf  jnri  v.f'J  in  di/fnulf  situahoni 

•  VtOv’pstes  -hinges  ini  is  prepared  t«  react 
acc  -tdirglv 


•  vej,  tr,_.j'5'.f  a  wide  rarge  v.f  difficult 

;  •'-rT>u^rca•-  ns  situations 

■  hm  extra  eff  r*  int-<  i imr  well 

•  1  apablf  'f  reMuad'^e  an  ludience 

•  *A  nr  ten  rep  -rts  can  be  easily  followed  by  iU 

f  j  JfM 

•  •  -mmu-.:  ati  ~s  ace  r.;c'ir.;»  ;  d  mere 

n,’.n"g  .  th-Sf  Ck.-:d»  necfiaiv  !0'sr,f’' 


•  Trac'  .'>  .1*  i  ic  t'dy  promotes  SF  primes  among 
pe":!  ar- *■  \  ;h  '•cJinates 

•  Fhjrurs  *r«  dr-elopmenri  an  |  afp.jej  them  to 
e»  is’irg  p"  "lures 

•  Ai'i'ej.  pr  nv'tei  par': .  ipsfi  »n  anl  wtiingh 
t  . ep*s  ,r'bs  ‘.hat  ni.hers  a»<-id 

•  E ra f irg  beha*rir  and  grooming  cteaie  a  »-rv 
fjv  ta^'e  impression 

•  Demr*n«*rjtm  exceptional  '-kill  in  working  with 

•then  a"d  rL'itang  thru  c-».p< •  ation 


WELL  ABOVE  S1ANOARO  (Ail  Hams) 

P«>ucuei  supeib  idhruiaJ  and  prufeuional 
k  n'  .wledpe 

Sufficiently  well  versed  in  his  or  her  j«'b  to  discuss 

and  implement  improved  meihodi  resulting  u> 

uvvgs  In  manpower  nr  material 

Maintains  and  Increases  professional  and  technical 

knowledge 

Actively  pursues  new  rdexs  and  developments  and 
their  relation  to  the  overall  mission 
Recognized  authority  in  his  or  her  Held 

keen,  anal)  uraJ  thinker 

MaVei  accutate  decisions  under  Intense  prcuuic 
Extremely  effective  in  exercising  logic  in  bioad 
areas  of  responsibility 


•  Abie  to  anticipate  critical  events  and  makes  prior 
provisions  to  deal  with  them 

•  PUns  encompass  all  feasible  contingencies 

•  Extremely  effective  in  ^ti^ration  of  resources 


•  Extremely  effective  in  use  of  material 

•  ( onustentlv  sre<s  and  protects  w;>i  of  using 
existing  equipment 

•  Is  often  assigned  ro  diffiCuir  _id  important 
projects  where  Ir-ni'cd  reso-t^es  arc  *  significant 
factor 

«  Ccnsiatentl)  demonstrates  outstanding  initiative 
and  accepts:  c  of  resp.mxihil.ty 

•  Exhibits  complete  confidence  tn  fus  or  her  ability 
to  handle  any  usk 

•  Induces  maoinium  effort  :rom  e.-erone 

•  Is  decisive  in  critical  sit-a'aons 

•  Pro-ujes  direction  ar.d  guidance  for  bruad  arias  of 
respor.sibd.it'.. 

•  leadership  not  limited  ’j  subord. rules  and  peers 

•  Re-pcnds  quickly  *.nd  c:fe:tiv.'l  to  c*vas 

•  S>  s'ematic  ally  vjc  e^ds  v  here  others  fa  ji 

•  fonustenth-  pr?vii'?s  o-tst:nd.ng  i'*.sd?:ihip  and 
guid.xntc  under  jiflicult  end  s’rcssf  J  conditioni 


•  [)e‘j*ers  cnn-:<e.  * e”-.  rp imerd  rrfsers»a*iiT.s 

•  l\  often  ci!!c>J  -n  • p.-esem  and  '“■plain  li.fHr 
;nd  v  -nplex  suhi'.  *s 

•  (  an  xwjv  j  hostile  a-ltence  to  bus  j:  her  point  ■( 
*i?w 

•  Able  to  describe  mplex  cr  lr<  J  ntcal  concrp'i  so 

we!!  that  c. en  t!’t  i-.ual  ’cade»  :sn  readily 

t  •■'ir.prrhend  trie  \  \<  \ 

•  li  rsijtentlv  !s  f  •  'be  mi-sr  imp.<'*ant  and 
difficult  writing  jss  p'.  :nrs 

•  It  frequently  iskf.1  *o  -<'•  :hr  written 
;  srresp.mdeni.e  i  f 

•  firm.  *a;t.  tnd  cs;'  rm  m  e~f  rung  Air  Torce 
C  ti : !r s  ’o  beaiirf .  bfva-v“?,  .o  v*o  -oung 

•  Itrprcsenti  l.:s  r  her  :;ani;jt:.-n  arJthe  Ab  f'-r 
the  most  ins  pot  tan  even’s 

•  V 'ive'v  prom’':rs  ■  :ga-.:r jtm-aJ  and  AF 

•blfCTlvcs 

•  rearing. behavior  a-  1  pt ' -mng  are  ’  uts’ -nding  ■ 

•  pe  nonstr.stes  rlearlv  “jpeti!  r  ability  to  work  with 
r  then  and  to  elicit  rheu  cooperation 
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10  •  Tre  rs  all  prnom?| 'arfi »  and  "juiuMy 

•  V  .luxtjjdy  part  cxpstfi  in  sctintin  in 
support  of  equal  eppof  urutv 

•  Shows  concern  and  u  seniitive  to  needs  of 
others 


•  f  i  i^b^M  i- i  'riK’ii  s'ii  i'Jt  ’san'xins 
I’andards  of  »quaJ  «“ppof»vn:’v 

•  F  ocour-Sges  pru'tce  of  equal  oppoir  imn  a;:d 
uea'ment  m  «J  activioei 

•  Citpljvi  a  high  drsnee  of  sensitivity  *rd  o  ncem 
frr  o*beis 


•  Activeli  rlein  -".stfjtes  s'rong  uible  and  credible 
j.  j-prfl  rif  equjj  -pp  ’Muniy 

•  Ls  extrem*lv  knowledgeable  in  the  area  of  equal 
''ppottunitv  and  trea'neni 

•  Display  s  ex  L'fme  sensitivity  and  a  deep  concern  In 
all  deilings  with  peers  mj  luU'fdinires 

•  Is  nxtrrmely  ellfiii.e  m  lofnng  human  relations 
problems  -  solutions  always  reflect  fau  and  equal 
ireaunent 
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